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MFT pon a wt> SYSTEM FOR TRANSMITTING 
MEDIA INFORMATION THROUGH A NETWORK 



This application claims priority pursuant to 35 U.S.C. §119 from 
Provisional Patent Application Serial No. 60/1 16,555 filed January 21, 1999, the 
1 0 entire disclosure of which is hereby incorporated by reference. 

FIELD OF THE INVENTION 
The present invention relates to transmitting media through a network, 
and in particular transmitting information specifying particular media or portions of 
15 media in order to govern and synchronize the rendition of such media. 

BACKGROUND OF THE INVENTION 
Internet technology affords users across the globe the ability to 
communicate, e.g., by electronic mail. The sender types or otherwise inputs a 
20 message into their personal computer and directs the message to a recipient. An 
application on the sender's computer establishes communication with a server 
connected to the Internet and transmits the message. The message may be sent from 
one server to another depending on the recipient's address. Finally, the destination 
server transmits the message to the recipient's personal computer. The message may 
25 include any electronic data ranging from simple text to complex audio-video material. 
The entire process can be completed in a very short time. 

With appropriate applications, such as ICQ and AOL's Instant 
Messaging and depending on network traffic and performance, the communication 
can be so fast as to seem instantaneous to the users participating in the 
30 communication. This is achieved by transmitting the message as it is being inputted 
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rather than waiting for the complete message to be input and then sending it. The 
basic requirement for this type of communication is that both of the users have 
activated access to the Internet (i.e. both users must be logged on to the network). In 
addition, the message is usually limited to textual data. To the users, the effect is real- 
5 time dialog using electronic data. This system is referred to as Instant Messaging. 

An extension of direct communication between the sender and a 
specific recipient or group of named recipients is dynamic group communication. 
This system, referred to as Chat, is a conversation among several users where 
participants can join or leave the conversation at any time. Chat groups may be open 

10 to the public or restricted, e.g., limited to persons with a password. 

There is a need for the application of advanced communication to 
multi-media data. What is further needed is a system that transmits data sequentially 
but also has synchronization capabilities so that users in different locations can 
experience the same media at the same time. The present invention satisfies these and 

15 other needs. 

SUMMARY OF THE INVENTION 
The present invention is a method and system for transmitting media 
information through a network in order to specify particular media or portions of 
media and to govern and synchronize the rendition of such media. Media objects 

20 stored on a server or other device connected to the network are accessible by 

specifying a content reference using an application appropriate for the network. For 
example, a standard browser is appropriate for accessing media stored on an Internet 
server where the content reference is a uniform resource locator. To send a media 
object from one user to another, the sender need only send the content reference. 

25 Conventional methods simply send only the location of the content. In the present 

invention, the content reference along with supplementary information is packaged in 
a data structure called a Handle to facilitate rendition of the media. A Handle may be 
sent to another user by E-mail, Chat, Instant Messaging, Cell Phone protocols or 
TV/Video links. When the recipient is ready to render the media object referenced by 
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the Handle, the recipient accesses the Handle and activates the appropriate software 
application such as a media or multi-media player. The software application provides 
an interface for the user to play the content. The Handle contains all the information 
needed to download content, and if applicable complete any commercial transactions 
5 pertaining to the use of the content. Specifically, the Handle includes information 
identifying each participant in the value chain, i.e., any entity that participated in the 
creation, resolution or transmission of the content that might receive some 
compensation for their participation. 

Typically, the player acquires the media object referenced by the 

10 Handle by downloading it from the network. Then the player facilitates the rendition 
of the content, producing the audio and displaying the graphics or video, etc. The 
rendition may be subject to commercial limitations such as purchase of rights to use 
the content, in which case, the player or another application may facilitate the 
commercial transaction. 

1 5 The player may also synchronize the rendition of the content at each of 

the users' locations. Using the supplemental information contained in the Handle, the 
rendering application, e.g. player, can coordinate playing the same content at the same 
time and rate at multiple locations. 

The present invention is also applicable where the media is stored 

20 locally, for example, on disk or DVD, or on the user's personal computer, e.g., having 
been previously downloaded from the Internet. 

BRIEF DESCRIPTION OF THE DRAWINGS 
Fig. 1 is a block diagram of an arrangement of a Handle data structure for 
25 implementing the present invention; 

Fig. 2 is a flow chart showing a method for using the Handle in accordance 
with the preferred embodiment of the present invention; and 

Fig. 3 is a flow chart showing an alternative method for using the Handle in 
accordance with an alternative embodiment of the present invention. 
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DETAILED DESCRIPTION OF THE 
PREFERRED EMBODIMENT OF THE INVENTION 

In the preferred embodiment of the present invention, e-mail and chat 
5 communication systems utilize Handles to provide linkages to content and 

synchronization of that content. Synchronization refers to playing the same content at 
multiple locations simultaneously such that each of the users is experiencing the same 
portion of content at the same time. The content referred to may be any multi-media 
data including, for example, text, graphics, music and music videos. The invention has 

10 application to all media, with musical content being a preferred embodiment and not 
limiting of the scope of the present invention. 

By way of overview and introduction, musical content, like other 
electronic data, may be distributed and transmitted over a network, such as the 
Internet. Musical content tends to require large amounts of data and the more 

1 5 complex the content, such as video material, the more voluminous the data. Because 
of the volume, it is not efficient to transmit the content in its entirety for every 
instance of reference. Instead of transmitting the content, only a reference to the 
content is transmitted. When the user receives the reference, the user can access the 
content directly. The content reference is referred to as a Handle and is discussed in 

20 detail below. 

To support the electronic distribution of musical content and other 
media over the Internet, for example, software applications may be implemented at 
the user's personal computer. A media player is one such application and provides an 
interface for the user to play the music. Musical content commercially distributed 

25 over the Internet may be placed in a secure format to prevent unauthorized use. For 
such secured content, the player effectively decrypts or otherwise processes the 
content in preparation for playing. For example, the music may be encrypted in such a 
way as to prevent playing unless a payment is made to the retailer or content owner 
for the use of the music. The player can assist the user in paying for the use of the 

30 music and then playing the music accordingly, for example, by interfacing with a 
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payment clearinghouse and executing a payment transaction, as is well known in the 
art. The terms "play" or "render" with reference to the content include anything that 
can be done to or with the content, such as producing audio, displaying visual content, 
printing, and copying, etc. 

The player may be used with content received over the Internet as well 
as locally stored content, i.e., stored on local or portable memory including diskettes, 
hard disks, optical disks, flash cards etc. The term disk is used throughout the 
description to refer to any such local or portable memory and is not intended to limit 
the scope of the invention. For content on disks, the player coordinates with the 
appropriate device such as a CD player or CD drive when rendering the content. 

When the user engaging in e-mail or chat communications wishes to 
transmit content to another user, the user need not actually transmit the content itself 
but may instead send a reference to the content, namely a Handle, so that the recipient 
may access the same content. 

Referring to Fig. 1, a Handle 100 is a small relatively secure data 
structure identifying particular content, and may contain various additional 
information about the content referenced. Content such as text, audio, graphics, 
video, etc, is stored in data structures called Content Objects and an Object ID 1 10 
may be included in the Handle as a reference uniquely identifying particular content. 
The Object ED is essential for the effective implementation of a Handle. However, 
where a group of content objects are associated, a single Handle may contain more 
than one Object ID each referencing one content object. 

Conceptually, each content object refers to an object in a product such 
as an album, single track, or EP (extended play). To identify the particular product 
containing the object, a numbering system such as the stock keeping unit (SKU) may 
be used. In such a case, the Handle would then include a SKU ID 1 12. 

As applied in the commercial environment where content is sold and 
distributed, the Handle identifies the content and all the participants of the value chain 
associated with the content. Together, the Object ID and SKU ID identify precisely 
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what the content is. Other identifiers may also be included in the Handle to identify 
each of the value chain participants. Participants include, for example, the artist, the 
retailer, the network provider, the consumer, the software player vendor, the device 
manufacturer or licensor, the patent holder, etc. The Distributor ID 1 14 identifies the 
owner of or the agent of the owner of the Content Object(s) referenced. The Retailer 
ID 1 16 identifies a Retailer associated with the content referenced in the Handle. This 
may be the Retailer from whom the content was purchased, or any Retailer that the 
Distributor (content owner) wishes to reference. For video, the Channel ED (not 
shown) of a network may be more appropriate than a Retail ID. The Channel ED may 
be, for example, HBO or ABC. The Renderer ID (not shown) refers to the software 
that created the Handle, such as the Real Jukebox, which may participate in the value 
chain (e.g. 50 basis points for transactions that resulted from a purchase that resulted 
from a Handle it created). The Carrier ED (not shown) may refer to anyone who is 
actually responsible for carrying the content. For example, the Carrier ID may be 
AT&T if it was delivered over a telephone network or it could be the SD Memory 
Association, to effect payment to the patent pool allowing the memory format which 
supports superdistribution. 



related to the content, such as a purchase. An application that dynamically computes 
offers for the sale or rental of content may use the value chain information in 
conjunction with a database of commercial information to generate the offers. One 
such application may be a reference service which maintains a database of commercial 
information from various participants of the value chain regarding the content. For 
example, a retailer may have contracted with a distributor for a 2% cut of the price 
provided the price is at least 7% above manufacturer cost and the retailer may have a 
mark up of 10% above cost. When a user wants to purchase particular content, the 
reference service can use the information in the Handle, such as the Retailer ID, to 
determine a valid offer based on the commercial information available from (or 
regarding) that retailer and provide the user with such an offer. The terms of an offer 
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may be included in the Handle as well. For example, the user may be presented with 
an offer to play the video for $2 per use anytime for a period of one year. Price, 
expiration date, and to whom payment is to be made are examples of terms that may 
be included in the Handle. 

The concept of a Handle is flexible and can refer to content stored 
locally or remotely. If the content is stored remotely, it is accessible through 
connection to a network such as the Internet. Where the content is accessible through 
the Internet, the Object ID may be a Uniform Resource Locator (URL). Content 
stored locally includes any kind of medium such as CD, DVD, Flash memory, and 
hard drive. For locally stored content, a Disk ID 1 18 may be included in the Handle. 

Typically, Handles are processor generated (e.g. by a computer or 
consumer device) when they are sent, but they can be stored locally or on a server and 
retrieved as needed. When a user retrieves a Handle and wishes to send it to another 
user, a User ID 120 may be included in the Handle to identify the user who is now 
sending the Handle. 

To facilitate synchronization (described in more detail below), the 
Handle may contain the time when the Handle is sent and information about the 
rendition of the content at the sender's location. For example, the Handle may include 
a Local Time ID 122 and an Absolute Time ID 124. The Local Time ID is the local 
time as known by the device rendering the content. The Absolute Time ID is the 
absolute time as known by the network, e.g. GMT. The Handle may also include a 
location marker to indicate a particular point or place in the content. For example, if 
the content is a video, the marker may be set to a particular scene, so that the scene 
may be referenced directly. Such information is contained in a Temporal Location ED 
126 which refers to a position, in the temporal domain, of the object referenced. For 
example, a Temporal Location ID may be expressed in units of time, e.g., 1 minute: 
23 seconds, or alternatively units of frames, e.g., 18 frames. In addition to marking a 
place in the content it is useful to note the state of the content, such as play or pause. 
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The state of the content may be included in the Handle in the form of Temporal State 
ID 128. 

Handles may be specialized for specific environments or applications. 
For example, Handles may be customized to create Network Handles which facilitate 
the electronic distribution of media over a network environment and the rendition of 
that media. A Disk Handle may be created to facilitate rendition of media stored 
locally. In addition, Handles may be customized to create Synch Handles which 
facilitate the synchronization of the rendition of the media in multiple locations. Each 
of these three examples is discussed below. 

Network Handles usually contain the basic information needed to refer 
to, acquire and consume Content Objects that have been electronically distributed 
over a network. A Network Handle would typically include the SKU ID, Distributor 
ID, Retailer or Channel ID, Renderer ID, User ID and some number of Object IDs. 

A user can attach a Network Handle in any number of ways, such as 
menu access, drag and drop, etc. Referring to Fig. 2, in a typical player environment 
the sequence is as follows: At step 210, the user clicks on a song title or other display 
element, which refers to a Content Object. At step 212, the user drags the selection 
into an e-mail message. At step 214, the player software binds together all the 
identifiers into the Network Handle. The identifiers include the Object ID(s), SKU 
ID, the User ID and identifiers for each of the participants of the value chain, e.g., 
Distributor ID, Retailer ID and Renderer ID. At step 216, the Handle is placed into 
the e-mail message. At step 218, the e-mail is sent to the recipient. At step 220, the 
recipient reads the e-mail and opens or accesses the Handle. At step 222, the e-mail 
application communicates with the operating system to call the application, which the 
user has designated to resolve Handles, usually, a media player application. The 
player may assist the user in acquiring and rendering the content. At step 224, the 
media player determines whether the content is resident locally. Typically, the content 
may be resident locally if the user previously acquired the content and stored it 
locally. If the content is not resident, at step 226, the player uses the Handle to 
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remotely access and download the content stored on some network server. Then at 
step 228, the media player serves as a user interface to facilitate the rendition of the 
content. The player uses the Handle (possibly in conjunction with other information 
such as commercial terms set by a retailer for that content) to determine the range of 
uses for which the user is authorized and/or has paid. 

Disk Handles work in a similar fashion as Network Handles. The 
typical sequence is as follows: User clicks on a song title or other display element 
which refers to content on the disk. User drags the selection into an e-mail message. 
The player software binds together the Object ID(s), SKU ID, User ID, and Disk ED 
for the content (e.g., song). The player also binds the Retailer Id and identifiers for 
other value chain participants. The Disk Handle is placed into the e-mail message and 
the e-mail is sent to the recipient. The recipient then reads the e-mail and opens or 
accesses the Handle, which results in the e-mail application communicating with the 
operating system to call the application, which the user has designated to resolve 
Handles, usually, a player application. The player renders the content (if it has the 
right to) and if the disk is available (inserted, attached, on network). If the content is 
not available, the user is given the choice of either inserting the disk (or connecting to 
the network) or going to a retail web site (on the Internet or other network) to 
purchase the disk or its electronic equivalent. 

Synchronization Handle (Synch Handle) is a specialized Handle that 
can be used in networked environments to synchronize two or more Content Objects 
that have temporal characteristics. To create a Synch Handle, the player application 
typically binds together the Temporal Location ID, Temporal State ID,the Local Time 
ID, the Absolute Time ID and the Object ID into a Handle that can be attached to or 
inserted in an electronic communication (e.g. chat window). The Synch Handle may 
be either a Network Handle or a Disk Handle with temporal information (the 
Temporal Location ED, the Temporal State ED, the Local Time ED, and the Absolute 
Time ID). The temporal information is used to synchronize temporal rendition (e.g. 
playing audio or video) when engaging in a dynamic chat. 
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Referring to Fig. 3, a sample usage scenario of the Synch Handle for a 
disk is as follows: At step 310, the user clicks on a song title or other display element 
which refers to content on the disk. At step 312, the user drags this into an e-mail 
message. At step 314, the player software binds together Object ID(s), SKU ID, Disk 
5 ID, User ID and identifiers for the value chain participants, e.g. Distributor ID, and 
Retailer ID. Additionally, for synchronization it adds the Temporal Location ID, the 
Temporal State ID, the Local Time ED and Absolute Time ID to the Handle. At step 
„ 316, this Handle is placed into the e-mail message. At step 3 1 8, the message is sent to 

%2 or seen by (as in chat environments) the recipient(s). At step 320, the recipient opens 

£0 10 or accesses the Handle. At step 322, the e-mail application communicates with the 
J ; j operating system to call the application which the user has designated to resolve 

tr| Handles, usually, a player application. At step 324, the player renders the content (if it 

g" has the right to), and if it is accessible (usually stored locally). 

*i The player resolves the objects temporally in the following manner: At 

C3 15 step 326, the player subtracts the Absolute Time ID in the Handle (when the Handle 
f* s was created) from the current absolute time to find the amount of time lapsed between 

5 3 the instantiation of the Handle and its resolution. The result is the Transport Time. At 

step 328, the player takes the Temporal Location ID (where in the object (song) the 
sender was when they sent it) and adds the Transport Time to determine where in the 
20 object (song) the sender is now. At step 324, the player renders the object beginning 
at that time according to the Temporal State ID (e.g. play). 

For example, assume that the sender and recipient each have the 
content resident locally and that it takes eight seconds from sending the e-mail until 
the recipient receives it. The sender begins to play the content and then decides to e- 
25 mail the recipient to synchronize playing the content. By the time the recipient 

receives the e-mail, the sender has experienced eight seconds of the content. Hence 
the recipient's player will start playing the content an additional eight seconds into the 
content to that it is perfectly synchronized with the sender's experience of the content 
with respect to an absolute time. 



// 



11 

An Affinity Group or Chat session refers to various communications 
between users through the same network including one-to-one communication, 
one-to-many communication, moderated or un-moderated group communication, 
Instant Messages, etc. In the context of Chat there can be multiple membership 
affinities based upon user defined preferences. Users can be members of multiple 
Affinity Groups and each of these Groups can be controlled independently and 
simultaneously in terms of privacy and availability parameters. Examples of some 
group definition are as follows: the Engineering Group available for Chat between 
9:00 AM and 5:00 PM, Monday through Friday; a group of close family members 
available for Chat at all times; an Online Gaming Group available for Chat whenever 
the user is playing a game on line (a user's preferences allow the user to filter the 
group for a specific game or any game the user is playing); a No Doubt fan club 
available for Chat whenever the user is listening to No Doubt; a Foreign Film Group 
available for Chat whenever the user is on a film site registered by the user or the film 
site. 

Dynamic Chat includes the ability to make the user available to an 
Affinity Group based upon user activity or external events. For example, a user plays 
a piece of music through a device (PC, DVD Audio Player, Interactive TV, Handheld 
Device, etc.) with an online connection (telephone, cable, cellular, satellite, etc). 
Once the music begins to play (either by inserting a disk into a drive or playing a 
stored file), the user is made available to a Dynamic Affinity (Chat) Group based upon 
the music they are listening to. A screen prompt (depending upon user preference) 
may be displayed asking the user if they would like to chat with others currently 
listening to the same or similar music. Participation in the Affinity Groupings may be 
overlapping because users may participate in multiple groups. The user can determine 
the basis or metric defining the group. For example the user may choose to chat with 
people listening to the same track, album, artist, or genre. If the members of the chat 
are interested in synchronizing the music they are listening to, a Synch Handle can be 
dragged from the player (see above) into the Chat Window. When the recipient sees 
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(receives) the Synch Handle, the recipient can activate (double-click) it and their 
music will be synchronized. If the recipient does not have the content or if it is on a 
disk not currently inserted, they are prompted to insert the disk or acquire the music. 

User availability can also be influenced (dependent upon subscriber 
5 preferences) by subscription or usage information. The user can look, for example, 
for others watching a television program or a particular movie, others interested in 
recipes or a particular sport, or others in a certain situation or geographic location. 
The categories, Televison, Movies, Recipes, Sports, etc, serve as metrics to assist in 
forming Affinity Groups. 
10 Web sites and Chat windows can interact in a number of ways. While 

browsing a web site, a user can become available to anyone else browsing that site or 
anyone browsing that site who shares any of the user's selection of metrics. 

Technical information or support for various purposes may be 
^ facilitated by the use of Handles. For example, if the user requires technical 

1 5 assistance regarding a product or feature, a reference or pointer to the source for such 
technical information or support may be included in the Handle. When such technical 
information is needed, the reference in the Handle may be used to access and 
download the information. Alternatively, all or part of the technical support 
information may be included directly in the Handle. A user may access technical 
20 support on one occasion and keep the information for future reference by storing it 
locally. Once information is stored locally, it may be updated by using the reference 
in the Handle to locate and download the updated information. 

While the invention has been particularly shown and described with 
reference to a preferred embodiment thereof, it will be understood by those skilled in 
25 the art that various changes in form and details may be made therein without departing 
from the spirit and scope of the invention. 



